{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 01 kNN 基础 k邻近算法(分类问题)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "简单 效果好 不用数学"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### kNN 基础概念\n",
    "\n",
    "k=3 就是新来一点 看离他最近的3个点 是属于什么类型 最多的那个类型 就是最可能的类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 实现我们自己的 kNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 创建简单测试用例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data_X = [[3.393533211, 2.331273381],\n",
    "              [3.110073483, 1.781539638],\n",
    "              [1.343808831, 3.368360954],\n",
    "              [3.582294042, 4.679179110],\n",
    "              [2.280362439, 2.866990263],\n",
    "              [7.423436942, 4.696522875],\n",
    "              [5.745051997, 3.533989803],\n",
    "              [9.172168622, 2.511101045],\n",
    "              [7.792783481, 3.424088941],\n",
    "              [7.939820817, 0.791637231]\n",
    "             ]\n",
    "raw_data_y = [0, 0, 0, 0, 0, 1, 1, 1, 1, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = np.array(raw_data_X) # 转np数组\n",
    "y_train = np.array(raw_data_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3.39353321, 2.33127338],\n",
       "       [3.11007348, 1.78153964],\n",
       "       [1.34380883, 3.36836095],\n",
       "       [3.58229404, 4.67917911],\n",
       "       [2.28036244, 2.86699026],\n",
       "       [7.42343694, 4.69652288],\n",
       "       [5.745052  , 3.5339898 ],\n",
       "       [9.17216862, 2.51110105],\n",
       "       [7.79278348, 3.42408894],\n",
       "       [7.93982082, 0.79163723]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFGxJREFUeJzt3X9s5Hed3/HnaxNXYOA21a1b0mxs94/o1EIooVagpEUR\nWyoSEqgq/ggyIPHPNDSqQnsSKlgCpZL/OKlCK4hE5BLKRcyBroRDJAon6JIr8EfCeZclmx+oSq/x\nJml68UFx2BrdbeDdP2Y2eI29Htvj/Y6/+3xIo5n5zGfn+8oq89rvfH/MN1WFJKldDjQdQJI0fJa7\nJLWQ5S5JLWS5S1ILWe6S1EKWuyS1kOUuSS1kuUtSC1nuktRClze14EOHDtX09HRTi5ekfen48eN/\nVVUTW81rrNynp6dZXFxsavGStC8lWRpknptlJKmFLHdJaiHLXZJayHKXpBay3CWphSx3SWohy12S\nWshyl7Q3ul2YnoYDB3r33W7TiS4plrsA6J7qMn10mgN3HWD66DTdU34QtQvdLnQ6sLQEVb37TseC\nv4gsd9E91aXzQIellSWKYmllic4DHQteOzc3B6ur54+trvbGdVFY7mLu2ByrZ8//IK6eXWXumB9E\n7dDp09sb19BZ7uL0ysYfuM3GpS1NTm5vXENnuYvJgxt/4DYbl7Y0Pw/j4+ePjY/3xnVRWO5i/sg8\n42PnfxDHx8aZP+IHUTs0OwsLCzA1BUnvfmGhN66LorGf/NXomL2294GbOzbH6ZXTTB6cZP7I/Cvj\n0o7MzlrmDUpVNbLgmZmZ8vfcJWl7khyvqpmt5rlZRpJaaOByT3JZkh8leXCD125MspLkZP/2qeHG\nlCRtx3a2ud8JPAX8ziavf7+qbtl9JEnSbg205p7kMPAe4At7G0eSNAyDbpY5Cnwc+PUF5rw9yWNJ\nvpXkDRtNSNJJsphkcXl5ebtZJUkD2rLck9wCvFhVxy8w7QQwWVVvAj4HfGOjSVW1UFUzVTUzMTGx\no8CSpK0NsuZ+A/DeJM8AXwXemeTLaydU1UtVdab/+CFgLMmhYYeVJA1my3Kvqk9U1eGqmgZuA75b\nVR9cOyfJ65Ok//j6/vv+dA/ySpIGsOMzVJPcDlBV9wDvBz6a5GXgl8Bt1dTZUZIkz1CVpP3EM1Ql\n6RJmuUtSC1nuktRClrsktZDlLkktZLlLUgtZ7pLUQpa7JLWQ5S5JLWS5S1ILWe6S1EKWuyS1kOUu\nSS1kuUtSC1nuktRCA5d7ksuS/CjJgxu8liSfTfJ0/yLZbxluTEnSdmxnzf1O4KlNXrsJuKZ/6wCf\n32UuSdIuDFTuSQ4D7wG+sMmU9wH3Vc8jwBVJrhxSRknSNg265n4U+Djw601evwp4ds3z5/pjkqQG\nbFnuSW4BXqyq47tdWJJOksUki8vLy7t9O0nSJgZZc78BeG+SZ4CvAu9M8uV1c54Hrl7z/HB/7DxV\ntVBVM1U1MzExscPIkqStbFnuVfWJqjpcVdPAbcB3q+qD66Z9E/hw/6iZtwErVfXC8ONKkgZx+U7/\nYJLbAarqHuAh4GbgaWAV+MhQ0kmSdmRb5V5Vfwb8Wf/xPWvGC7hjmMEkSTvnGaqS1EKWuyS1kOUu\nSS1kuUtSC1nuktRClrsktZDlLkktZLlLUgtZ7pLUQpa7JLWQ5S5JLWS5S1ILWe6S1EKWuyS1kOUu\nSS1kuUt7oduF6Wk4cKB33+02nUiXmEEukP2qJD9M8uMkTyS5a4M5NyZZSXKyf/vU3sSV9oFuFzod\nWFqCqt59p2PB66IaZM39r4F3VtU/At4MvLt/ndT1vl9Vb+7f/uNQU0r7ydwcrK6eP7a62huXLpIt\nL7PXv4Temf7Tsf6t9jKUtK+dPr29cWkPDLTNPcllSU4CLwLfqapHN5j29iSPJflWkjcMNaW0n0xO\nbm9c2gMDlXtV/aqq3gwcBq5P8sZ1U04Ak1X1JuBzwDc2ep8knSSLSRaXl5d3k1saXfPzMD5+/tj4\neG9cg3GH9K5t62iZqvo58DDw7nXjL1XVmf7jh4CxJIc2+PMLVTVTVTMTExO7iC2NsNlZWFiAqSlI\nevcLC71xbc0d0kOR3ib1C0xIJoCzVfXzJK8Gvg38QVU9uGbO64G/rKpKcj3wNWCqLvDmMzMztbi4\nOJT/CEktMj3dK/T1pqbgmWcudpqRk+R4Vc1sNW+QNfcrgYeTPAb8Ob1t7g8muT3J7f057wceT/Jj\n4LPAbRcq9t3onuoyfXSaA3cdYProNN1T/msutYo7pIdikKNlHgOu22D8njWP7wbuHm6039Y91aXz\nQIfVs73DzJZWlug80AFg9lq/8kqtMDm58Zq7O6S3ZV+doTp3bO6VYj9n9ewqc8c8flhqDXdID8W+\nKvfTKxt/LdtsXNI+5A7podhys8womTw4ydLKb39dmzzo1zWpVWZnLfNd2ldr7vNH5hkfO//r2vjY\nOPNH/LomSWvtq3KfvXaWhVsXmDo4RQhTB6dYuHXBnamStM6Wx7nvFY9zl6TtG+Zx7pKkfcZyl6QW\nstwlqYUsd0lqIctdklrIcpekFrLcJamFLHdJaiHLXZJayHKXpBbastyTvCrJD5P8OMkTSe7aYE6S\nfDbJ00keS/KWvYkrSRrEID/5+9fAO6vqTJIx4AdJvlVVj6yZcxNwTf/2VuDz/XtJUgO2XHOvnjP9\np2P92/pfG3sfcF9/7iPAFUmuHG5USdKgBtrmnuSyJCeBF+ldIPvRdVOuAp5d8/y5/tj69+kkWUyy\nuLy8vNPMkqQtDFTuVfWrqnozcBi4Pskbd7KwqlqoqpmqmpmYmNjJW0iSBrCto2Wq6ufAw8C71730\nPHD1mueH+2OSpAYMcrTMRJIr+o9fDbwL+Mm6ad8EPtw/auZtwEpVvTD0tJKkgQyy5n4l8HCSx4A/\np7fN/cEktye5vT/nIeAvgKeB/wz8mz1JO4K6p7pMH53mwF0HmD46TfdUt+lIkrT1oZBV9Rhw3Qbj\n96x5XMAdw402+rqnunQe6LB6dhWApZUlOg90ALyuq6RGeYbqLswdm3ul2M9ZPbvK3LG5hhJJUo/l\nvgunV05va1ySLhbLfRcmD05ua1ySLhbLfRfmj8wzPjZ+3tj42DjzR+YbSiRJPZb7LsxeO8vCrQtM\nHZwihKmDUyzcuuDOVEmNS+9Al4tvZmamFhcXG1m2JO1XSY5X1cxW81xzl6QWstwlqYUsd0lqIctd\nklrIcpekFrLcJamFLHdJaiHLXZJayHKXpBay3CWphQa5zN7VSR5O8mSSJ5LcucGcG5OsJDnZv31q\nb+JK0j7V7cL0NBw40Lvv7u1V27a8EhPwMvD7VXUiyeuA40m+U1VPrpv3/aq6ZfgRJWmf63ah04HV\n/sV9lpZ6zwFm9+aHBrdcc6+qF6rqRP/xL4CngKv2JI0ktdHc3G+K/ZzV1d74HtnWNvck0/Sup/ro\nBi+/PcljSb6V5A2b/PlOksUki8vLy9sOK0n70ulNrs622fgQDFzuSV4L3A98rKpeWvfyCWCyqt4E\nfA74xkbvUVULVTVTVTMTExM7zSxJ+8vkJldn22x8CAYq9yRj9Iq9W1VfX/96Vb1UVWf6jx8CxpIc\nGmpSSdqv5udh/PyrtjE+3hvfI4McLRPgXuCpqvrMJnNe359Hkuv77/vTYQaVpH1rdhYWFmBqCpLe\n/cLCnu1MhcGOlrkB+BBwKsnJ/tgngUmAqroHeD/w0SQvA78EbqumLvEkSaNodnZPy3y9Lcu9qn4A\nZIs5dwN3DyuUJGl3PEP1EtI91WX66DQH7jrA9NFpuqf29iQKSc0ZZLOMWqB7qkvngQ6rZ3vH2i6t\nLNF5oHcSxey1F++roqSLwzX3S8TcsblXiv2c1bOrzB3bu5MoJDXHcr9EnF7Z+GSJzcYl7W+W+yVi\n8uDGJ0tsNi5pf7PcLxHzR+YZHzv/JIrxsXHmj+zdSRSSmmO5XyJmr51l4dYFpg5OEcLUwSkWbl1w\nZ6rUUmnqXKOZmZlaXFxsZNmStF8lOV5VM1vNc81dklrIcpekFrLcJamFLHdJaiHLXZJayHKXpBay\n3CWphQa5EtPVSR5O8mSSJ5LcucGcJPlskqf7F8l+y97ElSQNYpCf/H0Z+P2qOpHkdcDxJN+pqifX\nzLkJuKZ/eyvw+f69JKkBW665V9ULVXWi//gXwFPAVeumvQ+4r3oeAa5IcuXQ00qSBrKtbe5JpoHr\ngEfXvXQV8Oya58/x2/8ASJIukoHLPclrgfuBj1XVSztZWJJOksUki8vLyzt5C0nSAAYq9yRj9Iq9\nW1Vf32DK88DVa54f7o+dp6oWqmqmqmYmJiZ2kleSNIBBjpYJcC/wVFV9ZpNp3wQ+3D9q5m3ASlW9\nMMSckqRtGORomRuADwGnkpzsj30SmASoqnuAh4CbgaeBVeAjw48qSRrUluVeVT8AssWcAu4YVihJ\n0u54hqoktZDlLkktZLlLUgtZ7vtc91SX6aPTHLjrANNHp+me6jYdSdIIGORoGY2o7qkunQc6rJ5d\nBWBpZYnOAx0AZq+dbTKapIa55r6PzR2be6XYz1k9u8rcsbmGEkkaFZb7PnZ65fS2xiVdOiz3fWzy\n4OS2xiVdOiz3fWz+yDzjY+PnjY2PjTN/ZL6hRJJGheW+j81eO8vCrQtMHZwihKmDUyzcuuDOVEmk\n98sBF9/MzEwtLi42smxJ2q+SHK+qma3mueYuSS1kuUtSC1nuktRClrsktZDlLkktNMhl9r6Y5MUk\nj2/y+o1JVpKc7N8+NfyYkqTtGOSHw74E3A3cd4E536+qW4aSSJK0a1uuuVfV94CfXYQskqQhGdY2\n97cneSzJt5K8YUjvKUnaoWH8nvsJYLKqziS5GfgGcM1GE5N0gA7A5KQ/biVJe2XXa+5V9VJVnek/\nfggYS3Jok7kLVTVTVTMTExO7XbQkaRO7Lvckr0+S/uPr++/5092+ryRp57bcLJPkK8CNwKEkzwGf\nBsYAquoe4P3AR5O8DPwSuK2a+jUySRIwQLlX1Qe2eP1ueodKSpJGhGeoSlILWe6S1EKWuyS1kOUu\nSS1kuUtSC1nuktRClrsktZDlLkktZLlLUgtZ7pLUQpa7JLWQ5S5JLWS5S1ILWe6S1EKWuyS1kOUu\nSS20Zbkn+WKSF5M8vsnrSfLZJE8neSzJW4YfU5K0HYOsuX8JePcFXr8JuKZ/6wCf330sSdJubFnu\nVfU94GcXmPI+4L7qeQS4IsmVwwooSdq+YWxzvwp4ds3z5/pjvyVJJ8liksXl5eUhLFqStJGLukO1\nqhaqaqaqZiYmJi7moiXpkjKMcn8euHrN88P9MUlSQ4ZR7t8EPtw/auZtwEpVvTCE95Uk7dDlW01I\n8hXgRuBQkueATwNjAFV1D/AQcDPwNLAKfGSvwkqSBrNluVfVB7Z4vYA7hpZIkrRrnqEqSS1kuUtS\nC1nukvaHbhemp+HAgd59t9t0opG25TZ3SWpctwudDqyu9p4vLfWeA8zONpdrhLnmLmn0zc39ptjP\nWV3tjWtDlruk0Xf69PbGZblL2gcmJ7c3Lstd0j4wPw/j4+ePjY/3xrUhy13S6JudhYUFmJqCpHe/\nsODO1AvwaBlJ+8PsrGW+Da65S1ILWe6S1EKWuyS1kOUuSS1kuUtSC1nuktRClrsktVB6F1JqYMHJ\nMrAEHAL+qpEQWzPbzoxqtlHNBWbbqVHNtpe5pqpqYqtJjZX7KwGSxaqaaTTEJsy2M6OabVRzgdl2\nalSzjUIuN8tIUgtZ7pLUQqNQ7gtNB7gAs+3MqGYb1Vxgtp0a1WyN52p8m7skafhGYc1dkjRkjZV7\nki8meTHJ401l2EySq5M8nOTJJE8kubPpTABJXpXkh0l+3M91V9OZ1ktyWZIfJXmw6SxrJXkmyakk\nJ5MsNp1nrSRXJPlakp8keSrJPxmBTL/X/7s6d3spyceaznVOkn/X/ww8nuQrSV7VdKZzktzZz/VE\nk39nTR7n/g7gDHBfVb2xkRCbSHIlcGVVnUjyOuA48C+r6smGcwV4TVWdSTIG/AC4s6oeaTLXWkn+\nPTAD/E5V3dJ0nnOSPAPMVNXIHROd5A+B71fVF5L8LWC8qn7edK5zklwGPA+8taqWRiDPVfT+3/+H\nVfXLJH8MPFRVX2o2GSR5I/BV4Hrgb4A/BW6vqqcvdpbG1tyr6nvAz5pa/oVU1QtVdaL/+BfAU8BV\nzaaC6jnTfzrWv43MTpMkh4H3AF9oOst+keQg8A7gXoCq+ptRKva+I8D/HIViX+Ny4NVJLgfGgf/d\ncJ5z/gHwaFWtVtXLwH8H/lUTQdzmvoUk08B1wKPNJunpb/Y4CbwIfKeqRiJX31Hg48Cvmw6ygQL+\nW5LjSTpNh1nj7wPLwH/pb876QpLXNB1qnduArzQd4pyqeh74T8Bp4AVgpaq+3WyqVzwO/LMkv5tk\nHLgZuLqJIJb7BSR5LXA/8LGqeqnpPABV9auqejNwGLi+/zWwcUluAV6squNNZ9nEP+3/vd0E3NHf\nLDgKLgfeAny+qq4D/h/wH5qN9Bv9zUTvBf5r01nOSfK3gffR+4fx7wGvSfLBZlP1VNVTwB8A36a3\nSeYk8Ksmsljum+hv074f6FbV15vOs17/q/vDwLubztJ3A/De/rbtrwLvTPLlZiP9Rn9tj6p6EfgT\nettER8FzwHNrvoF9jV7Zj4qbgBNV9ZdNB1njnwP/q6qWq+os8HXg7Q1nekVV3VtV/7iq3gH8X+B/\nNJHDct9Af8flvcBTVfWZpvOck2QiyRX9x68G3gX8pNlUPVX1iao6XFXT9L7Gf7eqRmJtKslr+jvG\n6W/y+Bf0vj43rqr+D/Bskt/rDx0BGt1xv84HGKFNMn2ngbclGe9/Vo/Q2y82EpL8nf79JL3t7X/U\nRI7Lm1goQJKvADcCh5I8B3y6qu5tKs86NwAfAk71t28DfLKqHmowE8CVwB/2j144APxxVY3UIYcj\n6u8Cf9LrAS4H/qiq/rTZSOf5t0C3vwnkL4CPNJwHeOUfwncB/7rpLGtV1aNJvgacAF4GfsQInBG6\nxv1Jfhc4C9zR1A5yz1CVpBZys4wktZDlLkktZLlLUgtZ7pLUQpa7JLWQ5S5JLWS5S1ILWe6S1EL/\nH4mgOoxccwX2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c2d6978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X_train[y_train==0,0], X_train[y_train==0,1], color='g')\n",
    "plt.scatter(X_train[y_train==1,0], X_train[y_train==1,1], color='r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "预测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAUoElEQVR4nO3df4zkd33f8edrfdvA8uNc4W3j+rw7rYKiNjgBunKglhDiaMUP27QNkYwWElCirRBpTBMJBVYCGWn/QIrIKUUCbTHFlCk/akjkQyaKY6DAH5juGduHOaS6rfe44NYbTM44S8gZ3v1j5uy99ezt7N7cfWe/+3xIq5n5fD8389L55uXvfr7f70yqCknS3jfRdABJ0mhY6JLUEha6JLWEhS5JLWGhS1JLHGjqha+44orqdDpNvbwk7UnHjh37q6qaHrStsULvdDqsrKw09fKStCclWd1qm0suktQSFroktYSFLkktYaFLUktY6JLUEha6JLWEhS5JLWGhS7o4ul3odGBionfb7TadqPUsdAHQPd6lc6TDxC0TdI506B73zacL0O3CwgKsrkJV73ZhwVK/yCx00T3eZeHoAqunVymK1dOrLBxdsNS1e4uLsL5+7tj6em9cF42FLhbvXmT9zLlvvvUz6yze7ZtPu3Ty5M7GNRIWujh5evCbbKtxaVszMzsb10hY6GLm4OA32Vbj0raWlmBq6tyxqaneuC4aC10sHV5iavLcN9/U5BRLh33zaZfm52F5GWZnIendLi/3xnXRNPbxuRof89f03mSLdy9y8vRJZg7OsHR46alxaVfm5y3wSyxV1cgLz83NlZ+HLkk7k+RYVc0N2uaSiyS1xNCFnuSyJN9K8oUB296aZC3Jff2f3x5tTEnSdnayhn4zcAJ4/hbbP1NVv3PhkSRJuzHUHnqSQ8DrgY9e3DiSpN0adsnlCPAu4GfnmfNrSR5IcnuSqwdNSLKQZCXJytra2k6zSpLOY9tCT3I98GhVHTvPtKNAp6p+GfgL4LZBk6pquarmqmpuenp6V4ElSYMNs4d+HXBjkoeBTwOvSvLJjROq6gdV9ZP+w/8E/PORppQkbWvbQq+qd1fVoarqADcBX6qqN2+ck+TKDQ9vpHfwVJJ0Ce36StEk7wdWquoO4HeT3Ag8CTwGvHU08SRJw/JKUUnaQ7xSVJL2AQtdklrCQpeklrDQJaklLHRJagkLXZJawkKXpJaw0CWpJSx0SWoJC12SWsJCl6SWsNAlqSUsdElqCQtdklrCQpeklrDQJaklhi70JJcl+VaSLwzY9nNJPpPkoST3JOmMMqQkaXs72UO/ma2/K/S3gB9W1S8AfwR84EKDSZJ2ZqhCT3IIeD3w0S2mvAG4rX//duBwklx4PEnSsIbdQz8CvAv42RbbrwK+B1BVTwKngRdsnpRkIclKkpW1tbVdxJUkbWXbQk9yPfBoVR0737QBY8/49umqWq6quaqam56e3kFMSdJ2htlDvw64McnDwKeBVyX55KY5p4CrAZIcAA4Cj40wpyRpG9sWelW9u6oOVVUHuAn4UlW9edO0O4Df7N9/Y3/OM/bQJUkXz4Hd/sEk7wdWquoO4FbgvyR5iN6e+U0jyidJGtKOCr2qvgJ8pX//vRvG/xb49VEGkyTtjFeKSlJLWOiS1BIWuiS1hIUuSS1hoUtSS1joktQSFroktYSFLkktYaFLUktY6JLUEha6JLWEhS5JLWGhS1JLWOiS1BIWunQxdLvQ6cDERO+22206kfaBXX/BhaQtdLuwsADr673Hq6u9xwDz883lUusN8yXRz0ryzST3J3kwyS0D5rw1yVqS+/o/v31x4kp7wOLi02V+1vp6b1y6iIbZQ/8J8KqqeiLJJPD1JF+sqm9smveZqvqd0UeU9piTJ3c2Lo3IMF8SXVX1RP/hZP/HL4CWtjIzs7NxaUSGOiia5LIk9wGPAndV1T0Dpv1akgeS3J7k6i2eZyHJSpKVtbW1C4gtjbGlJZiaOndsaqo3ruF4UHlXhir0qvppVb0YOARcm+RFm6YcBTpV9cvAXwC3bfE8y1U1V1Vz09PTF5JbGl/z87C8DLOzkPRul5c9IDqssweVV1eh6umDypb6tlK1s9WTJO8D/qaq/nCL7ZcBj1XVwfM9z9zcXK2srOzotSXtA51Or8Q3m52Fhx++1GnGTpJjVTU3aNswZ7lMJ7m8f//ZwKuB726ac+WGhzcCJ3Yf9/y6x7t0jnSYuGWCzpEO3eP+X1tqlR0eVHZ15mnDnOVyJXBbf897AvhsVX0hyfuBlaq6A/jdJDcCTwKPAW+9GGG7x7ssHF1g/UzvlLDV06ssHO2d3zt/jb/OSq0wMzN4D33AQWVP+T/XjpdcRmU3Sy6dIx1WTz/zP/TswVkefufDI0omqVGbWxp6B5UHHIfYj6szF7TkMk5Onh78K9dW45L2oB0cVPaU/3PtqUKfOTj4PN6txiXtUfPzvV3sn/2sd7vF+omn/J9rTxX60uElpibPPb93anKKpcOe3yvtR57yf649Vejz18yzfMMyswdnCWH24CzLNyx7QFTapzzl/1x76qCoJO13rTkoKknamoUuSS1hoUtSS1joktQSFroktYSFLkktYaFLUktY6JLUEha6JLWEhS5JLTHMNxY9K8k3k9yf5MEktwyY83NJPpPkoST3JOlcjLCSpK0Ns4f+E+BVVfUrwIuB1yR52aY5vwX8sKp+Afgj4AOjjSlJ2s62hV49T/QfTvZ/Nn+i1xuA2/r3bwcOJ8nIUkqStjXUGnqSy5LcBzwK3FVV92yachXwPYCqehI4DbxgwPMsJFlJsrK2tnZhySVJ5xiq0Kvqp1X1YuAQcG2SF22aMmhv/Bmfy1tVy1U1V1Vz09PTO08rSdrSjs5yqaq/Br4CvGbTplPA1QBJDgAHgcdGkE+SNKRhznKZTnJ5//6zgVcD39007Q7gN/v33wh8qZr65gxJ2qeG2UO/EvhykgeA/0FvDf0LSd6f5Mb+nFuBFyR5CPg94A8uTtzx0z3epXOkw8QtE3SOdOge7zYdSdI+dWC7CVX1APCSAePv3XD/b4FfH2208dc93mXh6ALrZ9YBWD29ysLRBQC/51TSJeeVohdg8e7Fp8r8rPUz6yzevdhQIkn7mYV+AU6ePrmjcUm6mCz0CzBzcGZH45J0MVnoF2Dp8BJTk1PnjE1NTrF0eKmhRJL2Mwv9AsxfM8/yDcvMHpwlhNmDsyzfsOwBUUmNSFOni8/NzdXKykojry1Je1WSY1U1N2ibe+iS1BIWuiS1hIUuSS1hoUtSS1joktQSFroktYSFLkktYaFLUktY6JLUEha6JLXEMF9Bd3WSLyc5keTBJDcPmPPKJKeT3Nf/ee+g55KkfavbhU4HJiZ6t93Rf7vZtt9YBDwJ/H5V3ZvkecCxJHdV1Xc2zftaVV0/8oSStNd1u7CwAOv9L8RZXe09Bpgf3Yf5bbuHXlWPVNW9/fs/Ak4AV40sgSS13eLi02V+1vp6b3yEdrSGnqRD7/tF7xmw+eVJ7k/yxSS/tMWfX0iykmRlbW1tx2ElaU86ucW3mG01vktDF3qS5wKfA95ZVY9v2nwvMFtVvwL8R+BPBz1HVS1X1VxVzU1PT+82syTtLTNbfIvZVuO7NFShJ5mkV+bdqvr85u1V9XhVPdG/fycwmeSKkSaVpL1qaQmmzv12M6ameuMjNMxZLgFuBU5U1Qe3mPPz/Xkkubb/vD8YZVBJ2rPm52F5GWZnIendLi+P9IAoDHeWy3XAW4DjSe7rj70HmAGoqo8AbwTenuRJ4MfATdXUVyFJ0jianx95gW+2baFX1deBbDPnQ8CHRhVKkrRzXim6j3SPd+kc6TBxywSdIx26x0d/YYOk5gyz5KIW6B7vsnB0gfUzvXNhV0+vsnC0d2HD/DUX99dASZeGe+j7xOLdi0+V+VnrZ9ZZvHu0FzZIao6Fvk+cPD34AoatxiXtPRb6PjFzcPAFDFuNS9p7LPR9YunwElOT517YMDU5xdLh0V7YIKk5Fvo+MX/NPMs3LDN7cJYQZg/OsnzDsgdEpRZJU9f/zM3N1crKSiOvLUl7VZJjVTU3aJt76JLUEha6JLWEhS5JLWGhS1JLWOiS1BIWuiS1hIUuSS1hoUtSSwzzFXRXJ/lykhNJHkxy84A5SfLHSR5K8kCSl16cuJKkrQzzeehPAr9fVfcmeR5wLMldVfWdDXNeC7yw//OrwIf7t5KkS2TbPfSqeqSq7u3f/xFwArhq07Q3AJ+onm8Alye5cuRpJUlb2tEaepIO8BLgnk2brgK+t+HxKZ5Z+iRZSLKSZGVtbW1nSSVJ5zV0oSd5LvA54J1V9fjmzQP+yDM+9auqlqtqrqrmpqend5ZUknReQxV6kkl6Zd6tqs8PmHIKuHrD40PA9y88niRpWMOc5RLgVuBEVX1wi2l3AL/RP9vlZcDpqnpkhDklSdsY5iyX64C3AMeT3Ncfew8wA1BVHwHuBF4HPASsA28bfVRJ0vlsW+hV9XUGr5FvnFPAO0YVSpK0c14pKkktYaFLUktY6Htc93iXzpEOE7dM0DnSoXu823QkSQ0Z5qCoxlT3eJeFowusn1kHYPX0KgtHFwCYv2a+yWiSGuAe+h62ePfiU2V+1vqZdRbvXmwokaQmWeh72MnTJ3c0LqndLPQ9bObgzI7GJbWbhb6HLR1eYmpy6pyxqckplg4vNZRIUpMs9D1s/pp5lm9YZvbgLCHMHpxl+YZlD4hK+1R6F3leenNzc7WystLIa0vSXpXkWFXNDdrmHroktYSFLkktYaFLUktY6JLUEha6JLXEMN9Y9LEkjyb59hbbX5nkdJL7+j/vHX1MSdJ2hvlwro8DHwI+cZ45X6uq60eSSJK0K9vuoVfVV4HHLkEWSdIFGNUa+suT3J/ki0l+aUTPKUnagVF8Hvq9wGxVPZHkdcCfAi8cNDHJArAAMDPjB0hJ0ihd8B56VT1eVU/0798JTCa5You5y1U1V1Vz09PTF/rSkqQNLrjQk/x8kvTvX9t/zh9c6PNKknZm2yWXJJ8CXglckeQU8D5gEqCqPgK8EXh7kieBHwM3VVOf+CVJ+9i2hV5Vb9pm+4fondYoSWqQV4pKUktY6JLUEha6JLWEhS5JLWGhS1JLWOiS1BIWuiS1hIUuSS1hoUtSS1joktQSFroktYSFLkktYaFLUktY6JLUEha6JLWEhS5JLbFtoSf5WJJHk3x7i+1J8sdJHkryQJKXjj6mJGk7w+yhfxx4zXm2vxZ4Yf9nAfjwhceSJO3UtoVeVV8FHjvPlDcAn6iebwCXJ7lyVAElScMZxRr6VcD3Njw+1R97hiQLSVaSrKytrY3gpSVJZ42i0DNgrAZNrKrlqpqrqrnp6ekRvLQk6axRFPop4OoNjw8B3x/B80qSdmAUhX4H8Bv9s11eBpyuqkdG8LySpB04sN2EJJ8CXglckeQU8D5gEqCqPgLcCbwOeAhYB952scJKkra2baFX1Zu22V7AO0aWSJK0K14pKkktYaFLUktY6JL2hm4XOh2YmOjddrtNJxo7266hS1Ljul1YWID19d7j1dXeY4D5+eZyjRn30CWNv8XFp8v8rPX13rieYqFLGn8nT+5sfJ+y0CWNv5mZnY3vUxa6pPG3tARTU+eOTU31xvUUC13S+Jufh+VlmJ2FpHe7vOwB0U08y0XS3jA/b4Fvwz10SWoJC12SWsJCl6SWsNAlqSUsdElqCQtdklrCQpeklkjvC4caeOFkDVgFrgD+qpEQ2zPb7oxrtnHNBWbbrXHNdjFzzVbV9KANjRX6UwGSlaqaazTEFsy2O+OabVxzgdl2a1yzNZXLJRdJagkLXZJaYhwKfbnpAOdhtt0Z12zjmgvMtlvjmq2RXI2voUuSRmMc9tAlSSNgoUtSSzRW6Ek+luTRJN9uKsNWklyd5MtJTiR5MMnNTWc6K8mzknwzyf39bLc0nWmjJJcl+VaSLzSdZaMkDyc5nuS+JCtN59koyeVJbk/y3f6/uZePQaZf7P9dnf15PMk7m851VpL/0P/3/+0kn0ryrKYznZXk5n6uBy/131mTFxa9AngC+ERVvaiREFtIciVwZVXdm+R5wDHgX1fVdxqORpIAz6mqJ5JMAl8Hbq6qbzQcDYAkvwfMAc+vquubznNWkoeBuaoau4tQktwGfK2qPprk7wFTVfXXTec6K8llwF8Cv1pVq2OQ5yp6/+7/WVX9OMlngTur6uPNJoMkLwI+DVwL/B3wZ8Dbq+p/XorXb2wPvaq+CjzW1OufT1U9UlX39u//CDgBXNVsqp7qeaL/cLL/MxZHtpMcAl4PfLTpLHtFkucDrwBuBaiqvxunMu87DPyvcSjzDQ4Az05yAJgCvt9wnrP+KfCNqlqvqieB/w78m0v14q6hbyNJB3gJcE+zSZ7WX9a4D3gUuKuqxiXbEeBdwM+aDjJAAX+e5FiShabDbPBPgDXgP/eXqj6a5DlNh9rkJuBTTYc4q6r+EvhD4CTwCHC6qv682VRP+TbwiiQvSDIFvA64+lK9uIV+HkmeC3wOeGdVPd50nrOq6qdV9WLgEHBt/9e8RiW5Hni0qo41nWUL11XVS4HXAu/oL/mNgwPAS4EPV9VLgL8B/qDZSE/rLwHdCPy3prOcleTvA28A/jHwj4DnJHlzs6l6quoE8AHgLnrLLfcDT16q17fQt9Bfn/4c0K2qzzedZ5D+r+ZfAV7TcBSA64Ab+2vVnwZeleSTzUZ6WlV9v3/7KPAn9NY4x8Ep4NSG37Jup1fw4+K1wL1V9f+aDrLBq4H/U1VrVXUG+DzwLxrO9JSqurWqXlpVr6C3rHxJ1s/BQh+of+DxVuBEVX2w6TwbJZlOcnn//rPp/eP+brOpoKreXVWHqqpD71f0L1XVWOw1JXlO/+A2/eWMf0XvV+PGVdX/Bb6X5Bf7Q4eBxg++b/Amxmi5pe8k8LIkU/336mF6x7nGQpJ/0L+dAf4tl/Dv78CleqHNknwKeCVwRZJTwPuq6tam8mxyHfAW4Hh/rRrgPVV1Z4OZzroSuK1/5sEE8NmqGqtTBMfQPwT+pPfe5wDwX6vqz5qNdI5/D3T7yxv/G3hbw3kA6K8B/0vg3zWdZaOquifJ7cC99JYzvsV4fQTA55K8ADgDvKOqfnipXthL/yWpJVxykaSWsNAlqSUsdElqCQtdklrCQpeklrDQJaklLHRJaon/Dy7CNUzXaJuHAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.array([8.093607318, 3.365731514]) # 新来一个数据 预测把\n",
    "\n",
    "plt.scatter(X_train[y_train==0,0], X_train[y_train==0,1], color='g')\n",
    "plt.scatter(X_train[y_train==1,0], X_train[y_train==1,1], color='r')\n",
    "plt.scatter(x[0], x[1], color='b')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### kNN的过程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt\n",
    "distances = [] \n",
    "for x_train in X_train: # 遍历每一个已知点 x_train\n",
    "    # 距离要考虑各个维度, 就是向量的每个值\n",
    "    d = sqrt(np.sum((x_train - x)**2)) # 距离差平方 之和 开根号 \n",
    "    distances.append(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4.812566907609877,\n",
       " 5.229270827235305,\n",
       " 6.749798999160064,\n",
       " 4.6986266144110695,\n",
       " 5.83460014556857,\n",
       " 1.4900114024329525,\n",
       " 2.354574897431513,\n",
       " 1.3761132675144652,\n",
       " 0.3064319992975,\n",
       " 2.5786840957478887]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 上面这堆 可以写成一行\n",
    "distances = [sqrt(np.sum((x_train - x)**2))  for x_train in X_train]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4.812566907609877,\n",
       " 5.229270827235305,\n",
       " 6.749798999160064,\n",
       " 4.6986266144110695,\n",
       " 5.83460014556857,\n",
       " 1.4900114024329525,\n",
       " 2.354574897431513,\n",
       " 1.3761132675144652,\n",
       " 0.3064319992975,\n",
       " 2.5786840957478887]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "distances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8, 7, 5, 6, 9, 3, 0, 1, 4, 2], dtype=int64)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nearest = np.argsort(distances) # 从小到大排序 返回坐标\n",
    "nearest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "k = 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "topK_y = [y_train[neighbor] for neighbor in nearest[:k]] # 最近的6点是什么结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 1, 1, 1, 0]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "topK_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "votes = Counter(topK_y) # 什么东西: 有几个"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({1: 5, 0: 1})"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "votes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1, 5)]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "votes.most_common(1) # 找出票数最多的1个元素  (元素:几个)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "predict_y = votes.most_common(1)[0][0] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "predict_y # 最终预测结果"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
